Hello!

Our Exchange infrastructure description, briefly:

1. Using Exchange 2013 CU7

2. 2 x MBX+CAS servers (EX1, EX2)

3. 2 x Edge servers (mx1, mx2)

4. DAG, 4 DBs

5. Hybrid Configuration: 200 local users, 15 cloud users

Our case: sometimes our users receive NDR while sending messages to existing mailboxes:

"Remote Server returned '550 5.1.1 RESOLVER.ADR.RecipNotFound; not found'

For example, user sent 2 messages, first caused NDR, second was successfully delivered:

Get-MessageTrackingLog -Sender sender@domain.ru -Recipient recipient@domain.ru | fl EventId, Source, TimeStamp, EventData

EventId   : HAREDIRECT
Source    : SMTP
Timestamp : 15.04.2015 16:58:36
EventData : {[DeliveryPriority, Normal], [AccountForest, local.domain.ru]}

EventId   : RECEIVE
Source    : SMTP
Timestamp : 15.04.2015 16:58:36
EventData : {[FirstForestHop, EX1.local.domain.ru], [Oorg, domain.ru], [ProxiedClientIPAddress, 10.0.100.21], [Proxie
            dClientHostname, mx1.domain.ru], [ProxyHop1, EX1.local.domain.ru(192.168.0.11)], [DeliveryPriority, Normal]
            , [AccountForest, local.domain.ru]}

EventId   : AGENTINFO
Source    : AGENT
Timestamp : 15.04.2015 16:58:36
EventData : {[AMA, SUM|action=st|error=|atch=0], [DeliveryPriority, Normal], [AccountForest, local.domain.ru]}

EventId   : FAIL
Source    : ROUTING
Timestamp : 15.04.2015 16:58:36
EventData : {[DeliveryPriority, Normal], [Oorg, domain.ru], [AccountForest, local.domain.ru]}

EventId   : HAREDIRECT
Source    : SMTP
Timestamp : 15.04.2015 18:17:23
EventData : {[DeliveryPriority, Normal], [AccountForest, local.domain.ru]}

EventId   : RECEIVE
Source    : SMTP
Timestamp : 15.04.2015 18:17:23
EventData : {[FirstForestHop, EX2.local.domain.ru], [ProxiedClientIPAddress, 10.0.100.22], [ProxiedClientHostname,
            mx2.domain.ru], [ProxyHop1, EX2.local.domain.ru(192.168.0.12)], [DeliveryPriority, Normal], [AccountForest,
            local.domain.ru]}

EventId   : AGENTINFO
Source    : AGENT
Timestamp : 15.04.2015 18:17:23
EventData : {[AMA, SUM|v=0|action=|error=|atch=0], [AMA, EV|engine=M|v=0|sig=1.195.3371.0|name=|file=], [TRA, ETR|ruleI
            d=068e0c20-5de5-48e0-9b53-ea90e5deb5b9|st=12.12.2014 18:27:31|action=SetSCL|sev=1|mode=Enforce], [CompCost
             |AMA=0|ETR=0], [DeliveryPriority, Normal], [AccountForest, local.domain.ru]}

EventId   : SEND
Source    : SMTP
Timestamp : 15.04.2015 18:17:25
EventData : {[E2ELatency, 4.863], [Microsoft.Exchange.Transport.MailRecipient.RequiredTlsAuthLevel, Opportunistic], [
            DeliveryPriority, Normal], [AccountForest, local.domain.ru]}

EventId   : DELIVER
Source    : STOREDRIVER
Timestamp : 15.04.2015 18:17:25
EventData : {[MailboxDatabaseName, IT], [Mailboxes, c1357fa5-52f2-4d16-8c70-09c9f9c4ab0e], [ToEntity, Hosted], [E2ELate
            ncy, 4.863], [DeliveryPriority, Normal], [AccountForest, local.domain.ru]}

How can I investigate this issue? 

• Edited by Turboon Friday, April 17, 2015 11:47 AM

Hello Ed, Jim!

Thank you for your useful answers!

We really have one corrupted DC in a branch site. One question before restoration:

How can we adjust Exchange settings to get data from DCs in certain site (head office site)?

Thanks in advance.

• Marked as answer by Turboon Tuesday, April 21, 2015 8:04 AM

Mappings are correct.

Exchange Servers are in 10.0.15.0/24 subnet, and head office subnet in AD Sites and Services is designated as 10.0.0.0/16.

Branch with corrupted DC mapped to 10.4.0.0/16 subnet.

• Edited by Turboon 21 hours 56 minutes ago

Mappings are correct.

Exchange Servers are in 10.0.15.0/24 subnet, and head office subnet in AD Sites and Services is designated as 10.0.0.0/16.

Branch with corrupted DC mapped to 10.4.0.0/16 subnet.

• Edited by Turboon Wednesday, April 22, 2015 9:31 AM

There's an Application event log message that shows up every 15 minutes that shows what DCs Exchange is using.  Take a look at that.

Yes, there are event log messages with source MSExchange ADAccess and EventId 2080 like this:

On site:
DC1.local.domain.ru CDG 1 7 7 1 0 1 1 7 1
DC2.local.domain.ru.ru CDG 1 7 7 1 0 1 1 7 1       
Out of site:
BRANCH-DC.local.domain.ru CDG 1 7 7 1 0 1 1 7 1

And sometimes Exchange servers connect out-of-site DCs for data.

• Edited by Turboon 23 hours 11 minutes ago

Yes, there are event log messages with source MSExchange ADAccess and EventId 2080 like this:

On site:
DC1.local.domain.ru CDG 1 7 7 1 0 1 1 7 1
DC2.local.domain.ru.ru CDG 1 7 7 1 0 1 1 7 1       
Out of site:
BRANCH-DC.local.domain.ru CDG 1 7 7 1 0 1 1 7 1

And sometimes Exchange servers connect out-of-site DCs for data.

• Edited by Turboon Thursday, April 23, 2015 8:16 AM

That says that the in-site DCs are healthy and can handle all tasks.  Exchange will use them unless there is a connectivity issue to them.

Thank you, Ed! :)